EVENT  DMAP  2002-011 


MILLENNIUM  CHALLENGE  2002 

DATA  MANAGEMENT  AND  ANALYSIS  PLAN 


1  July  2002 


DISTRIBUTION  STATEMENT  A 

Approved  for  Public  Release 
Distribution  Unlimited 


Joint  Interoperability 
Test  Command 


Prepared  by: 


Joint  Combat 
Identification 
Evaluation  Team 


and 


Single  Integrated  Air 
Picture  System  Engineer 


Document  Configuration  Control: 


Betty  Youmans 

Systems  Planning  and  Analysis,  Inc. 
4900  Seminary  Rd.,  Suite  400 
Alexandria,  VA  22311 
(703)  578-5696 


20020829  008 


EVENT  DMAP  2002-011 


MILLENNIUM  CHALLENGE  2002 

DATA  MANAGEMENT  AND  ANALYSIS  PLAN 


1  July  2002 


HojL-eJlfL.. 

Richard  Clarke 
JITC 

Col.  Gregory^.  Brown.  USAF 


Page  2 
1  July  2002 


EVENT  DMAP  2002-1 .1C.1.0 


MILLENNIUM  CHALLENGE  2002 

DATA  MANAGEMENT  AND  ANALYSIS  PLAN 


1  July  2002 


Approved  for  Public  Release: 
Distribution  is  Unlimited. 


Richard  Clarke 
JITC 


Col.  Gregory  M.  Brown,  USAF 
Commander,  JCIET 


CA 

US 

1 

U; 

1 

i 

ttery  W.  Wilson,  USN 
>TF 

Page  2 
Version  1. 1C. 1.0 
1  July  2002 


EVENT  DMAP  2002-1. 1C.1.0 


MILLENNIUM  CHALLENGE  2002 

DATA  MANAGEMENT  AND  ANALYSIS  PLAN 

1  July  2002 


Page  2 

Version  1. 1C.  1.0 
1  July  2002 


CAPT  Jeffery  W.  Wilson,  USN 
S!AP  $E  TF 


UKJf 


uouuui.  j  r  cj-t 


l-i— JWX  L_  I 


r  HUCL  oz 


EVENT  DMAP  2002-1 .10.1.0 

MILLENNIUM  CHALLENGE  2002 

DATA  MANAGEMENT  AND  ANALYSIS  PLAN 


1  July  2002 


Not  Yet  Approved  for  Public 
Release 


Richard  Clarke 
JITC 


y~\  - 

Col.  Gregory (Jm.  Brown.  USAF 


Brown.  USAF 


Commander,  JCIET 


CAPT  Joseph  A.  Horn,  Jr.,  USN 
USJFCOM 


CAPT  Jeffery  W.  Wilson,  USN 
SIAP  SE  TF 


Page  2 
Version  1. 1C.  1,0 
1  July  2002 


vf/wz/xm  z  u:j/ 


J1 Ho 


i-YVat 


EVENT  DMAP  2002-1.1C.1.0 

MILLENNIUM  CHALLENGE  2002 

DATA  MANAGEMENT  AND  ANALYSIS  PLAN 


1  July  2002 


Not  Yet  Approved  for  Public 
Release 


JigJLdJUL* 

Richard  Clarke 
JITC 


Col.  Gregory  M.  Brown,  USAF 
Commander,  JCIET 


CAPT  Joseph  A.  Horn,  Jr.,  USN 
USJFCOM 


CAPT  Jeffery  W.  Wilson,  USN 
SIAP  SE  TF 


Page  2 

Version  1. 1C.  1.0 
1  July  2002 


GOVERNMENT  RIGHTS  IN  DATA  STATEMENT 

Reproduction  of  this  publication  in  whole 
or  in  part  is  permitted  for  any  purpose 
of  the  United  States  Government 


Page  3 
1  July  2002 


EXECUTIVE  SUMMARY 


ISSUES 

Integrated  Air  Defense  System  (IADS)  performance  and  Joint  Data  Network 
(JDN)  assessments  must  be  conducted  to  determine  what  programs  need 
improvements.  Exercises  such  as  those  conducted  at  Millennium  Challenge  2002  are 
invaluable  for  determining  current  capabilities  and  projecting  future  requirements. 

BACKGROUND 

United  States  Joint  Forces  Command  (USJFCOM)  and  the  Single  Integrated  Air 
Picture  Task  Force  (SIAP  SE  TF)  have  requested  the  teaming  of  the  Joint  Combat 
Identification  Evaluation  Team  (JCIET),  the  Joint  Interoperability  Test  Command  (JITC), 
and  the  Single  Integrated  Air  Picture  Analysis  Team  (SAT)  to  develop  a  process  for 
determining  the  quality  and  means  for  improving  the  Single  Integrated  Air  Picture  (SIAP) 
during  various  field  events,  exercises,  and  experiments.  MILLENNIUM  CHALLENGE 
2002  (MC02)  will  be  used  to  formalize  the  relationships,  procedures,  and  methodologies 
required  to  establish  this  process.  After  MC02,  the  team  will  collectively  evaluate  the 
processes  and  procedures  used  to  incorporate  changes  as  necessary  to  support  future 
events,  such  as  ROVING  SANDS  2003. 

MC02  addresses  air  breathing  threat  (ABT),  cruise  missile  (CM),  and  ballistic 
missile  (BM)  tracks.  The  JDN  includes  these  tracks,  and  also  includes  ground,  surface, 
sub-surface,  and  space  tracks.  During  MC02,  JCIET,  JITC,  and  SAT  representatives 
will  collaborate  to  perform  an  assessment  of  the  JDN  targeted  specifically  at  the  SIAP. 

In  addition,  empirical  data  will  be  collected  to  support  SIAP  SE  TF  analysis  efforts  such 
as  perturbation  analysis  studies,  and  calibration  of  modeling/simulation  and  hardware- 
in-the-loop  tools. 

APPROACH 

This  document  is  in  part  a  companion  to  the  SIAP  Standard  Data  Management 
and  Analysis  Plan  (DMAP)  and  will  refer  often  to  that  document  for  background 
information.  The  JITC  MC  02  Interoperability  Support  Plan  is  also  a  key  reference 
document  for  background  on  JDN  assessment  processes  for  this  event. 

SCOPE 

This  document  provides  a  high-level  description  of  the  event  and  identifies  roles 
and  responsibilities  of  on-site  teams.  This  document  also  includes  details  of  planning 
efforts,  on-site  activities,  post-event  analysis  and  reporting,  and  lessons  learned 
processing. 
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1.  INTRODUCTION 


This  document  is  the  product  of  a  collaborative  effort  between  representatives  of 
the  Joint  Combat  Identification  Evaluation  Team  (JCIET),  the  Joint  Interoperability  Test 
Command  (JITC),  and  the  Single  Integrated  Air  Picture  Analysis  Team  (SAT).  In 
addition  to  establishing  the  business  processes  to  be  used,  this  document  will  serve  two 
primary  purposes.  The  first  purpose  is  to  describe  the  roles  and  responsibilities  of 
evaluation  organizations  in  support  of  United  States  Joint  Forces  Command 
(USJFCOM)  and  Single  Integrated  Air  Picture  System  Engineering  Task  Force  (SIAP 
SE  TF)  objectives.  The  second  purpose  is  to  identify  and  convey  MILLENNIUM 
CHALLENGE  2002  (MC02)  data-collection,  analysis,  and  reporting  requirements: 

1 .  Document  analysis  objectives  for  each  organization 

2.  Provide  data  collection  and  distribution  process  requirements  to  ensure 
accomplishment  of  analysis  objectives 

3.  Identify  instrumentation  required  for  data  collection  effort 

4.  Identify  points  of  contact  who  are  accountable  for  collection,  processing, 
reproduction,  distribution,  analysis,  and  reporting  of  data  within  each 
participating  organization 

5.  Define  the  process  and  schedule  for  data  analysis,  product  delivery,  test 
observation  report  processing,  lessons  learned  processing,  and  event 
reporting 

1.1  BACKGROUND 

United  States  Joint  Forces  Command  (USJFCOM)  and  the  Single  Integrated  Air 
Picture  Task  Force  (SIAP  SE  TF)  have  requested  the  teaming  of  the  Joint  Combat 
Identification  Evaluation  Team  (JCIET),  the  Joint  Interoperability  Test  Command  (JITC), 
and  the  Single  Integrated  Air  Picture  Analysis  Team  (SAT)  to  develop  a  process  for 
determining  the  quality  and  means  for  improving  the  Single  Integrated  Air  Picture  (SIAP) 
during  various  field  events,  exercises,  and  experiments.  The  MILLENNIUM 
CHALLENGE  2002  (MC02)  event  will  be  used  to  formalize  the  relationships, 
procedures,  and  methodologies  required  to  establish  this  process.  After  the  MC02 
event,  the  team  will  collectively  evaluate  the  processes  and  procedures  used  to 
incorporate  changes  as  necessary  to  support  future  events  such  as  ROVING  SANDS 
2003. 


MC02  addresses  air  breathing  threat  (ABT),  cruise  missile  (CM),  and  ballistic 
missile  (BM)  tracks.  The  JDN  includes  these  tracks,  and  also  includes  ground,  surface, 
sub-surface,  and  space  tracks.  During  MC02,  JCIET,  JITC,  and  SAT  representatives 
will  collaborate  to  perform  an  assessment  of  the  JDN  targeted  specifically  at  the  SIAP. 
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In  addition,  empirical  data  will  be  collected  to  support  SIAP  SE  TF  analysis  efforts  such 
as  perturbation  analysis  studies,  and  calibration  of  modeling/simulation  and  hardware- 
in-the-loop  tools. 

1.2  Roles  and  Responsibilities 

Table  1  lists  the  core  team  responsible  for  overall  planning  and  execution  of 
MC02  data  collection,  analysis,  and  reporting  efforts.  Pending  funding  and  availability, 
the  team  will  be  augmented  as  required  by  platform  operators,  subject  matter  experts 
representing  systems  that  participated  in  the  event,  and  functional  subject  matter 
experts  that  provide  expertise  in  a  specific  technical  area. 

Table  1 .  MILLENNIUM  CHALLENGE  2002  Points  of  Contact 


Organization 

Name 

Telephone 

Number 

(COMM) 

(DSN) 

E-mail 

(UNCLASSIFIED)  (SIPRNET) 

USJFCOM 

LCDR  Pat 

Bindl 

(757)  836-5869 

bindl@jfcom.mil 

USJFCOM 

Mr.  John 

Dorris 

(757)  836-5881 

dorrisj@jfcom.mil 

SIAP  SE  TF 

Maj.  Dave 
Chelen 

(703)  602-6441 
X252 

ChelenDE@NAVSEA.NAVY.MIL 

SIAP  SE  TF 

Mr.  Darrell 
Schultz 

(703)  602-6441 
X228 

SchultzDP@NAVSEA.NAVY.MIL 

SIAP  SE  TF 

Mr.  Joey 

Wang 

(703)  602-6441 
X215 

WangJ@NAVSEA.NAVY.MIL 

SIAP  SE  TF 

Christy 

DeHaven 

(540)  663-9730 

CDeHaven  @  northropgrumman.c 
om 

SIAP  SE  TF 

Betty 

Youmans 

(703)  578-5696 

eyoumans  @  spa.com 

JITC 

Maj.  George 
Elefteriou 

(520)  538-5052 
DSN  879-5052 

elefterg@fhu.disa.mil 
elefterg  @  honor.jitc.disa.smil.mil 

JITC 

Maj. 

Christopher 

Zimmerman 

(520)  533-0152 

zimmermc@fhu.disa.mil 

JITC 

Mr.  Thomas 
Cole 

(520)  533-5421 
DSN  821-5421 

colet@fhu.disa.mil 

JITC 

Mr.  John 

Dugas 

(520)  533-9257 
DSN  821-9257 

dugasj@fhu.disa.mil 

JITC  “i 

Aman  Adeli 

(520)  538-5474 

adelia@fhu.disa.mil 

JCIET 

Joe  Gordon 

(850)  882-6700 
(ext  7020) 

DSN  872-6700 
(ext  7020) 

joe.gordon@eglin.af.mil 

JCIET 

Jeff  Lutz 

(850)  882-6700 
X7500 

jeff.lutz@eglin.af.mil 
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NWAS 

Mr.  Dan 
Bergstrom 

(909)  273-5084 

BergstromDJ@corona.navy.mil 

CNA 

Mr.  Paul 
Symborski 

(703)  824-2424 

symborsp@cna.org 

PEO  AMD 

Mr.  Chuck 

T  reece 

(256)  864-7020 

Chuck.T  reece@peo.  mevatec.com 

1.2.1  USJFCOM 

The  Director,  Operational  Test  and  Evaluation  (DOT&E)  and  USJFCOM  are  co¬ 
sponsors  of  MC  02.  In  addition,  USJFCOM  is  responsible  for  identifying  any  Events  of 
Interest  (EOls)  for  analysis,  assisting  in  prioritization  of  post-event  analysis  focus  areas, 
and  providing  guidance  on  future  joint  analysis  collaboration  efforts. 

USJFCOM  is  also  responsible  for  providing  HLA  for  flying  objects  (ABTs,  CMs, 
and  BMs)  post  execution. 

1.2.2  Single  Integrated  Air  Picture  System  Engineer  Task  Force  (SIAP  SE  TF) 

The  SIAP  SE  TF  is  a  partner  with  USJFCOM  for  MC02  analysis  efforts  and  is 
responsible  for  identifying  data  and  analysis  requirements  needed  to  support  SIAP 
Block  1  issues.  Specifically,  the  SIAP  SE  TF  is  requesting  assistance  (time  and 
resources  permitting)  from  JITC,  JCIET,  and  SAT  representatives  to  support  IADS 
Performance  assessments,  root-cause  analysis,  SIAP  critical  experiment  evaluations, 
lessons  learned  reporting,  and  test  observation  report  efforts. 

1.2.3  JITC  Joint  Operations  C4I  Assessment  Team  (JOCAT) 

JITC  is  responsible  for  collecting  Link  16  data  via  a  Radio  Frequency  (RF) 
interface  using  a  Link  Monitoring  System  (LMS)-16,  range  data  from  the  Nellis  Air 
Combat  Training  System  (NACTS),  Tactical  Information  Broadcast  Service  (TIBS), 
Tactical  Receive  Equipment  and  Related  Applications  (TRAP)/TRAP  Data 
Dissemination  System  (TDDS)  data  via  the  Combined  Air  Operations  Center  (CAOC)  in 
Tactical  Data  Intercomputer  Message  Format  (TDIMF),  and  selected  voice  circuits. 

JITC  will  support  analysis  of  any  EOls  identified  to  JITC  by  USJFCOM  during 

MC02. 


JOCAT  will  record  Link  16,  NACTS,  TIBS/TRAP/TDDS,  and  selected  voice 
circuits  for  eight  hours  each  day  for  the  period  24  July  through  9  August  2002.  JITC  will 
time  stamp  anomalies  for  JCIET  and  JITC  post-event  analysis.  During  the  period  5  to  8 
August  2002,  the  execution  assessment  will  target  the  Nellis  AFB  live  event. 

USJFCOM  will  provide  high  level  architecture  (HLA)  for  flying  objects  (ABTs, 

CMs,  and  BMs)  post  execution.  JITC  will  integrate  HLA  into  JOCAT  at  that  point  for 
post  execution  JITC  analysis  use.  JITC  will  provide  JCIET  NACTS  and  HLA  data. 
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Finally,  representatives  from  JITC,  JCIET,  and  the  SAT  will  team  to  perform  root- 
cause  analysis  of  identified  SIAP  deficiencies. 

1.2.4  Joint  Combat  Identification  Evaluation  Team  (JCIET) 

JCIET  is  responsible  for  collecting  system  central  track  stores  and  host  link  data 
for  use  in  calculating  the  SIAP  attributes  as  defined  in  the  SIAP  SE  Technical  Reports 
2001  -001  and  2001-003  and  supporting  root  cause  analysis.  Section  3.3  of  the  SIAP 
Standard  DMAP  discusses  the  SIAP  attributes  and  the  IADS  performance  assessment 
process  in  further  detail.  JCIET  will  be  the  sole  repository  of  system  central  track  stores 
data  in  support  of  MC  02.  JCIET  will  coordinate  with  the  appropriate  materiel  developers 
and  program  managers  to  ensure  live  system  data  are  collected  during  the  time  period 
from  24  July  to  9  August.  JCIET  is  also  responsible  for  supporting  root  cause  analysis 
of  JDN-related  anomalies  identified  by  JITC. 

JCIET  and  JITC  organizations  will  individually  conduct  the  analysis  efforts  briefly 
described  above.  After  some  initial  data  analysis,  JITC  will  identify  to  JCIET  any 
requests  for  root  cause  analysis  support  based  on  JDN  level  observations  that  require 
analysis  of  system  level  data.  Also,  JCIET  will  identify  to  JITC  any  observations  or 
requests  on  MIL-STD  conformance  matters.  Representatives  from  the  SAT  will  provide 
assistance  as  required  for  these  efforts. 

1.2.5  Single  Integrated  Air  Picture  Analysis  Team  (SAT) 

Time  and  funding  permitting,  representatives  from  the  SAT  will  augment  the 
JCIET  and  JITC  efforts.  Specifically,  the  SAT  provides  subject  matter  experts  (pending 
availability)  to  assist  with  IADS  Performance  assessments,  root-cause  analysis,  SIAP 
critical  experiment  evaluations,  lessons  learned  reporting,  and  test  observation  report 
efforts. 


2.  MC  02  OVERVIEW 

MC  02  (MC  02)  is  a  joint  integrating  event  sponsored  by  U.S.  Joint  Forces 
Command  (USJFCOM),  bringing  together  both  live  field  exercises  and  computer 
simulation  24  July  -15  August  2002.  It  is  a  critical  building  block  of  future  military 
transformation. 

MC02  will  incorporate  elements  of  all  military  services,  most  functional/regional 
commands  and  many  DoD  organizations  and  federal  agencies.  The  Secretary  of 
Defense  has  directed  that  participants  involve  elements  representative  of  their  future 
force  concepts  such  as  the  Air  Force's  Expeditionary  Aerospace  Force,  the  Army's 
medium-weight  brigades  and  the  Navy's  "Forward  From  the  Sea'1  vision. 
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The  MC  02  assessment  has  two  phases  -  execution  and  post  execution.  JITC 
will  lead  the  execution  phase,  and  JCIET  will  lead  the  post  execution  phase.  Analysis 
will  be  performed  from  data  gathered  during  MC  02.  JITC  will  focus  its  analysis  efforts 
on  the  information  broadcast  onto  the  JDN,  JDN  management  and  operations  based  on 
CJCSM  61 20.01  B,  and  MIL-STD  601 6A  compliance.  JCIET  will  focus  its  analysis 
efforts  on  calculating  the  SIAP  attributes  as  defined  in  the  SIAP  SE  Technical  Reports 
2001-001  and  2001-003  using  system  level  recordings,  link  data,  and  TSPI.  JCIET  and 
JITC  will  team  to  perform  root  cause  analysis  of  identified  SIAP  deficiencies. 
Representatives  from  the  SAT  will  assist  in  the  IADS  performance  assessment  and 
root-cause  analysis  processes  discussed  in  Section  3.3.  and  3.4  of  the  SIAP  Standard 
DMAP  pending  time,  funding,  and  resource  availability. 


2.1  MC  02  JDN/SIAP  Goals 

The  primary  goals  for  MC02  are  to  establish  a  baseline  to  determine  the  quality 
of  the  SIAP  (and  eventually  the  entire  JDN)  and  to  provide  recommendations  for  its 
improvement.  Assessment  factors  are  as  follows: 

1 .  Provide  a  baseline  for  comparison  of  JDN  performance  during  future  USJFCOM 
and  SIAP  SE  TF  sponsored  field  events,  exercises,  and  experiments 

2.  Provide  a  baseline  for  comparison  to  measure  interoperability  progress  of  live 
and  simulated  systems 

3.  Identify  and  analyze  technical  interoperability  problems  (root  causes)  and  provide 
recommendations  for  improvement 

4.  Address  the  quality  of  the  SIAP  using  metrics  approved  by  the  SIAP  System 
Engineer  in  Technical  Report  2001-001 . 

5.  Support  the  life  cycle  of  developing  systems  as  specified  in  Chairman  of  the  Joint 
Chiefs  of  Staff  Instruction  (CJCSI)  621 2.01  B. 

2.2  MC  02  JDN/SIAP  Objectives 

The  objectives  of  the  MC02  event  are  as  follows: 

1.  Conduct  IADS  Performance  assessment,  root-cause  analysis,  and  applicable 
SIAP  Critical  Experiments  based  on  processes  detailed  in  the  SIAP  Standard 
DMAP 

2.  Conduct  assessment  of  MIL-STD  601 6A  and  CJCSM  61 20.01  B  compliance 

3.  Develop  a  consolidated  test  observation  report  (TOR)  and  lessons  learned 
reporting  procedure  to  consolidate  existing  JITC  and  SIAP  SE  TF  processes  to 
achieve  greater  synergy  of  effort  at  future  events 
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4.  Assess  the  extent  to  which  systems  participating  in  MC02  provide  data  and 
message  handling  services  for  effective  operation  of  a  Link  16  network 

5.  Establish  solid  working  relations  and  build  a  repeatable  process  through  teaming 
of  USJFCOM,  SIAP  SE  TF,  JITC,  JCIET,  and  SAT  representatives  in  preparation 
for  future  event  participation. 


2.3  MC  02  Products 

The  expected  products  for  MC  02  are  the  following: 

1 .  Evaluation  of  success  and  shortfalls  of  JITC/JCIET/SAT  coordination 
process  at  MC  02 

2.  Report  on  compliance  to  MIL-STD  601 6A  and  CJCSM  61 20.01  B 
(conducted  by  JITC) 

3.  Evaluation  of  SIAP  performance  based  on  SIAP  attributes  and  root-cause 
analysis  of  SIAP  deficiencies  (led  by  JCIET) 

4.  Identification,  documentation,  and  resolution  of  issues  and  deficiencies 
related  to  the  event,  in  particular,  limitations  of  Millennium  Challenge  as  a 
SIAP  venue 

5.  Refinement  of  standardized  processes,  tools,  and  collaborative  analysis 
methods 

6.  Analysis  of  applicable  SIAP  critical  experiments  in  support  of  Block  1 
issues  (led  by  SAT). 

2.4  MC  02  Schedule 

Figure  1  provides  a  schedule  for  the  MC  02  event. 
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ID 

Task  Name 

1 

Millennium  Challenge  2002 

2 

DMAP  Appendix  Development 

3 

DMAP  Appendix  Draft  Delivered 

4 

DMAP  Appendix  (Final) 

5 

Test  Readiness  Review 

6 

Spin  Up  Week 

7 

Event  Data  Collection 

8 

Analysis 

9 

Data  reduction  and  SIAP  attributes 
calculation 

10 

JlTC  Quick-Look  Report  on  execution 
analysis 

11 

Interchange  Meetings 

12 

Team  Analysis  Meeting 

13 

Team  Analysis  Meeting 

14 

Final  Report  with  JciET  and  JlTC 
briefing  to  USJFCOM 

May  1  Jun  |  Jul  1  Aug  [  Sep  |  Oct  [  Nov  j  Dec  |  Jan  |  Feb 


2003 


♦ 


Figure  1 .  MC  02  activity  schedule 


2.5  Event  Planning 

Table  2  provides  an  event  planning  worksheet  for  the  MC  02  event. 
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Table  2.  MC  02  Event  Planning  Worksheet 


Name  of  Event: 

Date(s): 

24  July  -15  Aug 

Location: 

Ft.  Irwin,  CA 

Type  (HWIL,  Live): 

Millennium  Challenge  2002 

2002 

Nellis  AFB,  NV 

Various  other  loc. 

Live,  Simulation 

Systems  participating: 


System  Name/Service 

Computer  Program 

Data  collection  type 

Participants  are  listed  in  Section  4.1.1. 

Version  (if  applicable) 

(Automated,  manual,  both) 

List  critical  experiments  to  be  conducted  (using  SIAP  Standard  DMAP  Chapter  3  designations).  Include  any 
discrepancies  between  how  experiment  will  be  conducted  and  the  experiment  description: 

Critical  experiments  2-10,  with  an  emphasis  on  7-9  and  possibly  10. 


Data  Extraction:  Provide  description  of  discrepancies  between  data  extraction  nodes  to  be  used  and  those 
described  in  Section  4.2.1  of  the  SIAP  Standard  DMAP. 


Model/Tool  Description  for  processing  and  analyzing  data  (i.e.  MSI,  built-in  analysis  tools): 
JCIET  computes  SIAP  attributes  and  identifies  SIAP-related  events  of  interest. 

JITC  has  a  set  of  analysis  tools  for  conducting  its  MIL-STD  601 6A  analysis. 
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3.  MC  02  ASSESSMENTS 


This  section  discusses  the  assessment  processes  of  activities  to  be  conducted  at 
Millennium  Challenge  02.  Section  3.1  discusses  the  MIL-STD  601 6A  and  CJCSM 
61 20.01  B  compliance  assessment  process.  The  IADS  assessment  process  is 
described  in  detail  in  the  SIAP  Standard  DMAP.  Highlights  of  the  process  are  provided 
in  this  document  for  reference  in  Section  3.2.  Finally,  Section  3.3  describes  the  analysis 
process  of  network  communication  performance  to  be  conducted  by  the  PEO  AMD,  as 
part  of  the  JCIET  Team. 

3.1  MIL-STD  601 6 A  and  CJCSM  61 20.01  B  Compliance 

JITC  will  determine  the  extent  to  which  the  systems  participating  in  MC  02 
provide  data  and  message  handling  services  for  effective  operation  of  a  Link  16 
network.  Systems  will  be  evaluated  on  whether  messages  comply  with  MIL-STD  601 6A 
and  their  system-specific  message  implementation  specifications  using  both  transmitted 
and  host  Link  16  data.  Additionally,  the  assessment  will  include  analysis  of  the  systems 
participating  in  the  JDN  to  comply  with  CJCSM  6120.01  B.  Simulated  systems  as  well 
as  live  systems  will  be  included  in  this  analysis  effort. 

JITC’s  real-time  and  quick  look  analysis,  JDN  characterization,  and  reporting 
thereon  are  described  in  JITC’s  Interoperability  Support  Plan  for  MC  02. 

3.1 .1  MIL-STD  601 6A  and  CJCSM  61 20.01  B  Compliance  Approach 

JITC  will  conduct  post-event  analysis  of  participating  systems’  compliance  with 
MIL-STD  601 6A  message  protocols  and  approved  Interface  Change  Proposals  using 
the  following  high-level  MIL-STD  functional  area  metrics: 

(a)  Assess  systems’  conformance  to  message  protocols  in  the 
System  Information  Exchange  and  Network  Management 

functional  area  consisting  of  the  following  message  set  (whenever 
present). 

1  JO.O  Initial  Entry 

2  J0.1  Test 

3  J0.2  Network  Time  Update 

4  JO.  3  Time  Slot  Assignment 

5  JO.  4  Radio  Relay  Control 

6  JO.  5  Repromulgation  Relay 

7  JO.  6  Communications  Control 

8  JO.  7  Time  Slot  Reallocation 

9  J1.0  Connectivity  Interroga tion 
10  J1. 1  Connectivity  Status 

J1.2  Route  Establishment 
12  J  1.3  Acknowledgement 
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13  J  1.4  Communicant  Status 

14  J1.5  Net  Control  Initialization 

15  J1.6  Needline  Participation  Group  Assignment 

Assess  systems’  conformance  to  message  protocols  in  the 

Precise  Participant  Location  and  Identification  (PPLI) 

functional  area  consisting  of  the  following  message  set  (whenever 
present). 

1  J2.0  Indirect  Interface  Unit  PPLI 

2  J2.2  Air  PPLI 

Assess  systems’  conformance  to  message  protocols  in  the  Air 
Surveillance  functional  area  consisting  of  the  following  message 
set  and  related  actions  (whenever  present). 

J3.2  Air  Track 

-  Track  Correlation 

-  Positional  Data 
Velocity 

-  Strength 
Track  Quality 

-  Reporting  Responsibility 

Assess  systems  conformance  to  message  protocols  in  the  Space 
Surveillance  functional  area  consisting  of  the  following  message 
set  and  related  actions  (whenever  present). 

J3. 6  Space  Track 

-  Ballistic  Missile  Track  Correlation 

-  Ballistic  Missile  Positional  Data 

-  Ballistic  Missile  Velocity 

-  Lost  Track  Indicator 

-  Time  for  Ballistic  Missile  Tracks 

-  Non-real  Time  Ballistic  Missile  Track  Report 

-  Ballistic  Missile  Reporting  Responsibility 

-  Ballistic  Missile  Information  Difference  Recognition 
and  Resolution 

-  Ballistic  Missile  Surveillance  Data  Difference 
(Position  and  Velocity) 

-  Termination  of  Ballistic  Missile  Track  Data 

-  Purging  of  Remote  Ballistic  Missile  Tracks 

Assess  systems’  conformance  to  message  protocols  in  the 
Electronic  Surveillance  functional  area  consisting  of  the 
following  message  set  and  related  actions  (whenever  present). 
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1  J3. 7  Electronic  Warfare  Product  Information 

-  Line  of  Bearing  (LOB)  Reporting 

-  Electronic  Warfare  Fix  Reporting 

-  Area  of  Probability  Reporting 

-  EW  Track  Reporting 

-  Time  Data 

2  J14.0  Parametric  Information  Message 

(f)  Assess  systems’  conformance  to  message  protocols  in  the 

Electronic  Warfare  (EW)  Intelligence  functional  area  consisting 
of  the  following  message  set  (whenever  present). 

X  J6.0  Intelligence  Information 

2  J14.2  Electronic  Warfare  Control/Coordination 

(g)  Assess  systems’  conformance  to  message  protocols  in  the 
Mission  Management  functional  area  consisting  of  the  following 
message  set  (whenever  present). 

X  J13. 0  Airfield  Status 

2  J13.2  Air  Platform  and  System  Status 

(h)  Assess  systems’  conformance  to  message  protocols  in  the 
Weapons  Coordination  and  Management  functional  area 
consisting  of  the  following  message  set  (whenever  present). 

1  J9.0  Command 

2  J9. 1  Engagement  Coordination 

3  J9.2  ECM  Coordination 

4  J10.2  Engagement  Status 

5  J10.3  Handover 

6  J10.5  Controlling  Unit  Report 

7  J1 0.6  Pairing 

(i)  Assess  systems’  conformance  to  message  protocols  in  the 
Information  Management  functional  area  consisting  of  the 
following  message  set  and  related  actions  (whenever  present). 

X  3.0  Reference  Point 

2  J3. 1  Emergency  Points 

3  J7.0  Track  Management 

-  Drop  Track 

ID  Difference  Report 

-  Change  Data  Order 

-  Emergency  Status  Change 
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-  Force  Tell  Status  Change 

-  Strength  Change 

-  Exercise  Status  Order 

4  J7. 1  Data  Update  Request 

5  J7.2  Correlation 

6  J7.3  Pointer 

7  J7.4  Track  Identifier 

8  J7.5  IFF/SIF  Management 

9  J7. 6  Filter  Management 

10  J7. 7  Association 

1 1  J8.0  Unit  Designator 

12  J8. 1  Mission  Correlator  Change 

13  J15.0  Threat  Warning 


(j)  Assess  systems’  conformance  to  message  protocols  in  the 

Control  functional  area  consisting  of  the  following  message  set 
(whenever  present). 


1  J12.0  Mission  Assignment 

2  J12. 1  Vector 

3  J12.2  Precision  Aircraft  Direction 

4  J1 2.3  Flight  Path 

5  J12.4  Controlling  Unit  Change 

6  J12.5  Target/T rack  Correia  tion 

7  J1 2.6  Target  Sorting 

8  J 12. 7  Target  Bearing 

9  J17.0  Weather  Over  Target 


3.2  Process  For  IADS  Performance  Assessment 
3.2.1  Critical  Experiments 

It  is  anticipated  that  the  participants  and  activities  planned  to  occur  during  MC  02 
will  satisfy  the  SIAP  critical  experiments  listed  below.  Following  the  MC  02  event  the 
SAT  will  determine  (time  and  finding  permitting)  which  vignettes  provided  sufficient 
participation  and  data  to  support  a  specific  critical  experiment.  The  SIAP  Standard 
DMAP  contains  a  comprehensive  description  of  each  critical  experiment. 

•  Data  Registration  -  Uncompensated  data  registration  errors  have  a  significant 
adverse  impact  on  the  SIAP.  Systems  must  implement  standardized  sensor 
registration,  aligning  sensors,  and  inertial  navigation  systems  to  a  common  geodetic 
reference. 
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•  Automatic  Local-to-Remote  Track  Correlation/Decorrelation  -  Not  all  systems 
implement  automatic  correlation/decorrelation  processing,  and  differences  in  the 
methods  employed  degrade  warfighting  capability. 

•  Identification  Processing  -  Mis-identification  is  frequently  caused  by  erroneous 
IFF/SIF-to-track  association.  Additionally,  differences  in  the  way  automated 
identification  and  data  fusion  systems  behave  adds  to  failure  to  achieve 
commonality. 

•  Formation  Tracking  and  Assessment  -  Can  systems  automatically  assign  the 
identification  of  a  track  that  has  been  declared  to  have  a  strength  greater  than  one  to 
the  tracks  within  the  prescribed  formation  “window.” 

•  Model  and  Simulation/Stimulation  Fidelity  -  Combination  of  modeling/simulation, 
HWIL  and  open-air  events  provide  sufficient  fidelity  to  reflect  operational  system 
performance  to  support  assessment  and  engineering  efforts  to  predict  warfighting 
improvements  of  implementing  ICPs. 

•  PPLI  Accuracy  -  PPLIs  are  believed  to  be  very  accurate;  however,  due  to  data  link 
latencies  and  the  fact  that  some  navigation  systems  are  not  integrated  with  the  data 
link,  inaccuracies  may  be  present.  These  inaccuracies  will,  if  present,  degrade  the 
network  navigation  solution  and  data  registration  performance. 

•  Multi-Link  Translation/Forwardinq  -  There  is  a  need  to  reliably  translate  and  forward 
information  from  one  tactical  data  link  to  another  tactical  data  link  in  ways  that 
support  the  SIAP.  It  is  uncertain  at  the  time  of  the  signing  of  this  DMAP  whether 
MC  02  will  be  able  to  support  this  experiment. 


3.2.2  Operational  Context 

Because  MC  02  is  a  live-fly  and  simulation  exercise  the  participant  forces 
operate  in  a  manner  similar  to  that  of  a  real  conflict.  As  such,  the  scenarios  and  flight 
profiles  are  dynamic,  unscripted  events.  In  order  to  relate  MC  02  events  to  the  SIAP 
Common  Reference  Scenario  (CRS)  the  SAT  will  review  selected  vignettes  (time  and 
funding  permitting)  and  determine  the  commonality  to  the  CRS. 

3.2.3  IADS  Performance  Assessment 

The  assessment  of  the  IADS  performance  will  be  conducted  by  calculating  the 
SIAP  attributes  and  performing  root-cause  analysis.  The  majority  of  the  performance 
assessment  analysis  will  be  conducted  post-event. 
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3.2.3.1  SIAP  Attributes 


The  SIAP  attributes  analysis  will  be  conducted  in  accordance  with  SIAP  SE  TF 
Technical  Reports  2001-001  and  2001-003  and  described  in  further  detail  in  Section 
3.3.  of  the  SIAP  Standard  DMAP.  This  analysis  will  take  place  post-event  and  will  be 
led  by  JCIET. 

3.2.3.2  Root-Cause  Analysis 

Root-cause  analysis  in  support  of  the  IADS  performance  assessment  will  be 
conducted  both  on-site  during  MC  02  and  post-event  and  will  be  led  by  JCIET. 


3.2.4  Post-Event  Analysis  Efforts 

Time  and  funding  permitting,  the  SAT  will  meet  after  the  MC  02  event  in 
conjunction  with  the  JCIET  analysts  at  the  JCIET  headquarters,  Eglin  AFB,  on  a 
periodic  basis  to  complete  the  IADS  performance  assessment  analysis  (SIAP  Metrics 
and  root-cause). 


3.3  Communication  Performance  Analysis 

The  PEO  AMD,  as  part  of  the  JCIET  Team,  will  conduct  an  analysis  of  network 
communication  performance.  An  overview  of  this  analysis  follows: 

3.3.1  Data  Collection  Overview 

PEO  AMD  will  collect  Link-16  data  and  coordinate  the  collection  of  other  system 
data  to  support  the  assessment  of  the  MC  02  Link-16  Assessment  Objectives.  The  PEO 
AMD  will  collect  Link-16  data  via  LMS-16  and  JADE  system(s)  located  at  FTCPAC  in 
San  Diego.  The  JADE  computer  will  record  the  Link-16  messages  received  by  the 
LMS-16  as  well  as  the  truth  data  (podded  aircraft).  PEO  AMD  will  also  conduct  some 
manual  data  collection  by  querying  major  participating  systems  to  determine,  if  possible, 
why  they  were  not  available  on  Link-16  during  the  course  of  the  day/exercise. 

At  the  end  of  each  day,  data  log  files  will  be  saved  and  archived  according  to 
plan  from  the  LMS-16  and  JADE  systems.  The  JADE  and  LMS-16  system  will  be  used 
to  support  AARs  with  selected  replay  of  data  and  the  near  real  time  analysis  of 
objectives,  as  time  permits.  Test  Observation  Reports  (TOR)  will  be  provided  to 
JCIET/JITC  for  any  anomalies  observed.  These  TORs  may  be  used  in  the  Final 
Analysis  Report  (FAR).  A  full  assessment  of  Link-16  Assessment  Objectives  will 
require  post-mission  analysis.  Details  of  the  analysis  planned  for  each  of  these 
objectives  is  provided  below. 
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3.3.1. 1  Objective  -  Assess  Communications  Performance 

This  objective  involves  an  assessment  of  Link-16  communications  performance 
for  both  the  local  radio  frequency  (RF)  network  and  the  JRE  using  STJ.  The  objective 
consists  of  four  sub-objectives: 

.  Assess  Link-1 6  Connectivity.  The  times  when  J2.xx  messages  were  received 
from  each  system  to  determine  a  measure  of  Link-1 6  connectivity. 

•  Assess  Track  Reporting  Capacity.  An  assessment  of  the  usage  of  allocated 
track  capacity  for  each  Joint  Tactical  Information  Distribution  System  (JTIDS) 
Unit  (JU)  will  be  made. 

•  Assess  Ability  To  Meet  Link-1 6  Throughput  Requirements.  Track  latency  and 
lost  track  messages  will  be  assessed  to  determine  how  well  the  JDN  handled 
message  loading. 

•  Assess  Track  Number  Block  Assignments.  Track  number  usage  was 
assessed  for  each  JU  capable  of  reporting  tracks. 

3.3.1. 1.1  Sub-Objective  1  -  Assess  Link-16  Connectivity 

J2.X  PPLI  messages  will  be  assessed  for  each  JU  to  determine  when  each 
system  was  active  on  Link-16.  A  plot  of  JU  availability  will  be  generated  for  each  JU. 
Outages  will  be  investigated  to  determine  the  cause. 

The  database  from  the  JADE  computer  in  the  TBD  will  be  used  to  generate  a 
report  on  when  each  JU  was  transmitting  PPLI.  Availability  plots  will  be  generated  from 
this  data  using  an  Excel  spreadsheet.  JICO  and  system  logs  (if  available)  indicating 
when  and  why  systems  were  not  participating  on  Link-16  will  be  examined. 

As  a  fallback,  the  reporting  of  J3.2  Air  Track  messages  by  JlADS  systems  during 
ABT  raids  was  used  as  an  indication  of  Link-16  connectivity.  Because  simulation  issues 
could  prevent  a  system  from  reporting  tracks,  this  is  not  an  exact  measure  of 
operational  availability.  It  also  does  not  distinguish  between  raids  when  Link-16  was 
stable  and  raids  when  only  a  few  track  messages  made  it  through.  It  is  used  here  only 
as  an  indication  of  the  degree  that  JlADS  systems  were  available  to  interoperate  as  a 
family  of  systems  during  MC  02.  The  primary  causes  preventing  systems  from 
participating  on  Link-16  will  be  made  from  observations,  as  well  as  contacting 
participating  organizations  to  determine  causes. 

An  investigation  will  also  be  conducted  to  determine,  if  possible,  the  root  cause  of 
High  Error  Rate  of  received  messages. 

3.3.1. 1.2  Sub-Objective  2  -  Assess  Track  Reporting  Capacity 

The  allocation  of  timeslots  for  track  reporting  for  each  JU  will  be  assessed.  A  plot 
of  allocated  timeslot  usage  versus  time  will  be  generated.  Periods  when  timeslot 
capacity  is  reached  for  a  given  JU  will  be  noted  and,  if  required,  changes  to  timeslot 
allocations  will  be  recommended. 
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The  LMS-16  All  Trends  plot  was  used  to  support  this  analysis.  This  plot  consists 
of  a  strip  chart  for  selected  systems  that  shows  the  number  of  timeslots  used  to  report 
tracks  (including  air  tracks,  space  tracks,  surface  tracks,  ground  tracks,  TBM  ELPs  and 
PGIPs)  on  the  Surveillance  Net  Participation  Group  (NPG).  Many  of  the  systems 
participating  in  the  exercise  had  tracks  forwarded  through  another  system.  The 
assessment  involved  direct  reporting  systems,  as  well  as  the  other  systems  that  were 
being  forwarded. 

3.3.1 .1 .3  Sub-Objective  3  -  Assess  the  Ability  to  Meet  Link-1 6  Throughput 
Requirements 

Track  messages  sent  and  received  by  participants  (when  available)  will  be  compared 
with  LMS-1 6  recordings  to  determine  how  many  messages  were  lost  or  delayed  due  to 
SATCOM  capacity  constraints  or  other  issues.  Periods  when  SATCOM  capacity  is 
exceeded  will  be  noted  and  correlated  to  the  track  loading  and  associated  scenario. 

LMS-16  recordings  and  JADE  recordings  at  FTCPAC  and  recordings  at  participants 
(when  available)  will  be  used  to  compare  times  when  selected  tracks  were  generated  at 
one  end  of  the  SATCOM  channel  and  when  they  were  received  at  the  other.  NOTE: 

This  sub-objective  can  only  be  assessed  if  SATCOM  or  JREs  are  used  in  the 
exercise. 


3.3.1 .1.4  Sub-Objective  4  -  Track  Number  Block  Assignments 

Track  number  usage  will  be  assessed  for  each  JU  capable  of  reporting  tracks. 
Instances  where  track  numbers  are  reused  frequently  will  be  noted  and,  if  required, 
changes  to  track  block  assignments  will  be  recommended.  The  Access  database 
generated  by  the  JADE  computer  at  FPCPAC  will  be  used  to  query  track  numbers 
reported  by  each  JU  and  the  reuse  of  these  track  numbers  will  be  assessed. 

3.4  Test  Observation  Reports 

JITC  and  JCIET  (using  PEO  AMD  assets)  will  generate  Test  Observation 
Reports  (TORs)  during  the  MC  02  event.  SAT  members  will  assist  in  the  TOR  process. 
These  TORs  document  a  perceived  anomaly,  point  of  interest,  incident  or  situation  that 
requires  further  analysis.  They  are  applicable  to  both  the  compliance  and  the  IADS 
performance  assessment  efforts.  Report  formats,  handling,  and  adjudication  of  TORs 
are  discussed  in  more  detail  in  Section  4.3.1 .1 .  Post  MC02,  the  team  will  review  the 
process  and  apply  lessons  learned  in  preparation  for  use  of  the  revised  TOR  process 
during  RS03. 

3.5  Perturbation  Analysis 

Time  and  funding  permitting,  the  SAT  will  conduct  perturbation  analysis  following 
the  MC  02  event  in  accordance  with  the  SIAP  Standard  DMAP. 
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4.  EVENT  EXECUTION  AND  DATA  REQUIREMENTS 


4.1  Test  Plan  and  Supporting  Documents 

This  document  will  serve  as  the  test  plan  for  this  event.  The  next  section  gives 
an  overview  of  the  MC  02  event  and  its  participants. 

4.1.1  MC  02  Scenario  and  Participating  Systems 

MC  02  examines  those,  and  other,  capabilities  the  U.S.  military  would  like  to 
have  around  2007.  Events  involving  a  potential  future  adversary  -  played  by  fictional 
"Country  X"  -  are  part  of  the  experiment's  crisis  scenario. 

About  80  percent  of  the  experiment  will  consist  of  table-  top  activities,  while  20 
percent  involves  troops  and  equipment.  Troop  activity  will  occur  at  Fort  Irwin,  Calif.;  air 
operations  will  be  conducted  at  Nellis  Air  Force  Base,  Nevada;  and  Navy  and  Marine 
activities  will  be  held  off  the  coast  of  California. 

4.1 .1.1  Participating  Agencies 

The  following  lists  the  non  service-specific  Commands  participating  in  MC  02: 

•  Joint  Forces  Command 

•  US  Special  Operations  Command 

•  US  Space  Command 

•  US  Transportation  Command 

4.1 .1.2  Participating  Systems:  Army 

The  following  lists  participants  from  the  US  Army: 

•  US  Army  Forces  Command 

•  US  Army  T raining  and  Doctrine  Command 

•  US  Army  Transformation 

•  Army  Special  Operations  Command 

•  US  Army  Space  and  Missile  Defense  Command 

•  III  Corps 

•  82nd  Airborne  Division 

•  325  Airborne  Infantry  Regiment 

•  3rd  Brigade  Combat  Team,  2nd  Infantry  Division 

•  1  -1 01  st  Aviation  Regiment,  1 01  st  Airborne  Division  (Air  Assault) 

•  US  Army  Communications  -  Electronic  Command 

•  Army  T ransformation  Experiment  2002 
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4.1.1 .3  Participating  Systems:  Navy 

The  following  lists  participants  from  the  US  Navy: 

•  Naval  Weapons  Station  China  Lake 

•  Fleet  Combat  T raining  Center  Pacific 

•  Naval  Air  Station  Paxtuxent  River 

•  Naval  Air  Station  Point  Mugu 

•  Naval  Station  Newport 

•  Naval  Amphibious  Base  Coronado/Naval  Air  Station  North  Island 

•  Naval  Surface  Warfare  Center  Dahlgren 

•  USS  BOXER 

•  USS  COMSTOCK 

•  USS  CORONADO 

•  USS  FITZGERALD 

•  USS  LAKE  ERIE 

4.1 .1.4  Participating  Systems:  USMC 

The  following  lists  participants  from  the  US  Marines  Corps: 

•  Marine  Forces  Atlantic 

•  Marine  Corps  Warfighting  Laboratory 

•  First  Marine  Expeditionary  Force 

4.1. 1.5  Participating  Systems:  Air  Force 

The  following  lists  participants  from  the  US  Air  Force: 

•  Air  Combat  Command 

•  Air  Mobility  Command 

•  Air  Force  Space  Command 

•  12th  Air  Force 

•  Air  Warfighting  Center  (AWFC),  Nellis  AFB 

•  Electronics  Systems  Center  (ESC),  Hanscom  AFB 
.  AWACS 

•  AOC 

•  JSTARS 

•  Rivet  Joint 


4.2  Test  Item  Description  and  Configuration 

During  the  spin-up  portion  (prior  to  the  MC  02  event),  it  is  imperative  that  each 
system  representative  ensures  that  the  data  extraction  points  listed  in  the  data 
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collection  matrices  are  working  and  providing  the  data  expected  for  conducting  the 
critical  experiments  (e.g.,  at  the  appropriate  data  rates,  are  recording  properly).  The 
SlAP  Standard  DMAP  provides  tables  in  Section  4  with  the  appropriate  data  extraction 
points  and  the  format  of  the  data  to  be  collected. 

4.2.1  Data  Extraction  Diagrams 

Participants  will  follow  the  data  extraction  diagrams  provided  in  the  SlAP 
Standard  DMAP. 

4.2.2  Data  Recording 

JITC’s  JOCAT  will  record  data  or  import  data  from  other  sources  using  the  file 
formats  defined  in  Table  3: 


Table  3.  JITC  JOCAT  Data  Recording 


Category 

Recording  Device 

File  Format 

Link  16 

LMS-1 6 

.sdx  (proprietary  Northrup 
Grumman  format) 

Link  16 

JITC’s  Theater  Missile  Defense 
(TMD)  Interoperability  Assessment 
Capability  (TIAC)  Link  16  Parser 

Comma  Separated  Variable 
(CSV) 

TIBS 

JITC’s  TIBS  Interface  Processor 

TDIMF 

TDDS 

JITC’s  TDDS  Interface  Processor 

TDIMF  1 

NACTS 

JITC’s  NACTS  Interface  Processor 

CSV 

Voice 

Eyretel  Voice  Recorder 

.wav  audio  files 

HLA 

HLA  results  (run  by  USJFCOM) 

Microsoft  Access  Database 
files 

4.2.3  Data  Collection  Requirements 

JITC  will  primarily  use  four  types  of  data  for  the  MIL-STD  assessment:  Link  16 
data  (collected  by  LMS-1 6),  participating  systems’  host  link  data,  ground  truth  for  live 
systems  in  Nellis  airspace  and  reported  by  NACTS,  and  ground  truth  for  simulated 
objects  via  HLA  recording.  JCIET  will  obtain  participating  systems’  host  link  data,  and 
with  necessary  approval  for  each  system  obtained,  will  provide  this  data  to  JITC. 

JCIET  will  calculate  the  SlAP  attributes.  The  SlAP  attributes  assess  the  quality 
of  the  track  information  the  operators  had  available  to  them.  Consequently,  the 
information  available  to  the  operators  (often  called  Central  Track  Stores,  or  CTS)  is 
required  for  this  analysis.  Furthermore,  the  form  of  the  system  data  used  for  this 
analysis  must  be  consistent  with  how  the  information  is  presented  to  the  operator.  This 
is  particularly  important  in  the  case  of  mutual  tracks,  where  both  local  and  remote  data 
are  available:  the  data  assessed  for  each  system  must  be  consistent  with  its  operator 
display  implementation.  In  most  cases,  calculation  of  the  SlAP  attributes  depend  on 
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assignment  of  tracks  to  ground  truth,  so  ground  truth  data  must  also  be  available.  Root 
cause  determinations  may  also  require  link  data  (from  all  participants).  For  the  CAOC, 
ADS  I  data  will  be  used  as  the  primary  data  for  SIAP  analysis,  supplemented  with 
TBMCS  SAA  data  for  certain  time  periods  of  interest. 

Each  system  representative  is  responsible  for  collecting  data  that  will  be  read 
into  the  PET  tool.  The  variables  and  formats  needed  for  PET  are  provided  in  Section 
4.2.2  of  the  SIAP  Standard  DMAP.  Table  4  summarizes  the  data  needed  to  support 
SIAP  attribute  calculations  and  root  cause  analysis.  JCIET  will  ensure  the  system  data 
identified  in  Table  4  is  collected  and  reduced.  A  more  explicit  table  of  extraction  points 
and  data  collected  will  be  provided  in  future  events  such  as  Roving  Sands  03. 


Table  4.  JCIET  Coordinated  Data  Recording 


System 

Extraction  Point 

Extracted  Data 

AOCP 

Central  track  file 

AWACS 

AOCP 

Data  link  input/output 

JTIDS  Terminal 

Data  link  input/output 

CAOC-ADSI 

ADSI  host 

Data  link  input/output 

TBD 

Central  track  file 

CAOC  -TBMCS 

TBD 

Data  link  input/output 

TBD 

Operator  Actions 

CSC 

Central  track  file 

CSC 

Data  link  input/output 

Rivet  Joint 

CSC 

Operator  Actions 

JTIDS  Terminal 

Data  link  input/output 

CSC 

Central  track  file 

JSTARS 

CSC 

Data  link  input/output 

CSC 

Operator  Actions 

JTIDS  Terminal 

Data  link  input/output 

USS  BOXER  (LHD  4) 

TBD 

Link  input/output  buffers 

Central  Track  Stores 

USS  CORONADO  (AGF  1 1 ) 

TBD 

Link  input/output  buffers 

Central  track  stores 

C&D 

Central  track  stores 

C2P 

Data  link  input/output 

USS  LAKE  ERIE  (CG-70) 

SGS 

SPY 

WCS  Mark  8 

USS  FITZGERALD  (DDG-62) 

TBD 

Link  input/output  buffers 

Central  track  stores 

CSC 

Track  file 

E-2C 

CSC 

Data  link  input/output 

CSC 

Operator  actions 

Link-16  at  SOCAL 

PEO  AMD 

IAW  PEO  AMD  plan 

|  *  Recorded  using  Terminal  recording  device  | 
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4.3  Data  Exchange 


JITC  will  provide  data  to  JCIET  from  the  LMS-16  recordings  as  .sdx  files.  JITC 
will  also  provide  JCIET  NACTS  data  in  csv  and  PET  tool  formats.  JCIET  will  be  the  sole 
repository  for  system  central  track  files  and  link  input/output  files.  After  JCIET  has 
obtained  the  necessary  approval  for  each  system,  it  will  provide  JITC  with  tactical 
systems’  host  link  data  (link  input/output  buffer  files)  specified  in  Table  4. 

4.3.1  On-Site  Analysis  Activity 

JITC  will  conduct  limited  on-site  data  analysis  to  support  daily  debriefings 
focusing  on  MIL-STD  601 6A  compliance  and  JDN  performance  during  the  experiment. 
JITC  will  assist  JCIET  by  providing  workspace  for  up  to  two  JCIET  analysts  during 
conduct  of  the  MC  02  experiment. 

JCIET  on-site  activity  will  focus  on  collecting  the  data  in  Table  4  as  well  as 
assisting  in  identifying  SIAP-related  events  of  interest.  These  include  misidentifications, 
mis-associations,  leakers,  incidents  of  fratricide,  IFF/SIF  differences,  dual  tracks,  and 
other  issues  discussed  in  the  SIAP  Standard  DMAP. 

One  of  the  goals  of  MC  02  is  to  establish  a  process  for  JITC,  JCIET,  and  the  SAT 
to  conduct  TOR  recording,  tracking,  and  adjudication.  Due  to  limited  funding  and 
resources  for  MC  02,  the  complete  TOR  process  (generating,  exchanging,  tracking  and 
adjudicating)  discussed  in  Section  4.3.1 .1  will  be  conducted  to  a  limited  degree  (1-5 
TORs)  to  demonstrate  and  assess  the  TOR  process.  This  is  a  prototype  process  that 
will  be  matured  to  meet  JCIET,  JITC,  and  SAT  requirements  for  future  events. 

Due  to  funding  and  availability,  on-site  SAT  participation  will  be  limited. 
Specifically,  representatives  of  the  SAT  will  be  on-site  to  exercise  the  new  consolidated 
TOR  process  in  conjunction  with  JITC  and  JCIET.  No  on-site  SAT  activity  will  take 
place  for  root-cause  analysis  or  for  daily  debriefs. 

4.3.1. 1  Test  Observation  and  Final  Analysis  Reports  (TORs/FARs)  Generation  and 
Management 

When  an  anomaly  is  observed  during  the  test  event  or  discovered  during  post¬ 
test  analysis,  the  analyst  will  document  the  anomaly  in  a  TOR  and  associated  FAR. 
JCIET,  JITC,  and  the  SAT  will  generate  TORs.  SAT  members  will  assist  in  the  TOR 
process.  All  TORs  written  during  the  event  will  undergo  a  screening  process.  Any  TOR 
that  is  incomplete  or  requires  additional  information  will  be  sent  back  to  the  originator  for 
further  clarification.  The  SAT  will  enter  those  TORs  that  are  deemed  SIAP-related  and 
analytically  relevant  into  a  tracking  table  for  later  SAT  analysis. 
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4.3.1 .1.1  TOR/FAR  Numbering  Scheme 


To  easily  identify  and  track  TORs,  JCIET  and  JITC  analysts  will  identify  TORs 
using  the  characters  “MC02”  and  a  sequential  three-or-four-digit  number  according  to 
the  following  block  numbering  scheme: 

•  The  TOR  numbers  (500-699)  are  reserved  for  the  SAT  to  document 
anomalies  noted  during  the  test. 

•  The  TOR  numbers  (700-899)  are  reserved  for  JCIET  personnel  for 
documentation  of  anomalies  noted  during  the  test  based  on  real-time 
observations. 

•  The  TOR  numbers  (900-1 099)  are  reserved  for  JITC  personnel  for 
documentation  of  anomalies  noted  during  the  test  based  on  real-time 
observations. 

•  The  TOR  numbers  (1100-1299)  have  been  reserved  for  JCIET 
documentation  of  anomalies  noted  during  the  IADS  performance  analysis 
process. 

•  The  TOR  numbers  (1300-1499)  have  been  reserved  for  JITC 
documentation  of  interoperability  anomalies  noted  during  the  JDN  analysis 
process. 

Example:  MC02- 1310  indicates  a  test  session  that  was  conducted  for  Millennium 
Challenge  02  and  it  was  the  tenth  TOR  written  by  JITC  during  the  test  session. 

FARs  associated  with  a  given  TOR  will  bear  that  TOR’s  tracking  number.  To 
support  multiple  FARs  on  a  given  TOR,  FARs  will  also  carry  an  incremental  version 
number  (i.e.,  MC02-1310-1,  MC02-1310-2,  etc...). 

JCIET  and  JITC  will  track  the  status  of  their  TORs/FARs  respectively,  using  an 
MS  Word  or  Excel  file  for  that  purpose.  These  files  should  be  regularly  updated  and 
exchanged  between  JCIET,  JITC,  and  the  SAT  via  SIPRNet  email  during  the  post-test 
analysis  phase  of  MC  02. 

4.3.1 .1.2  Post-Test  Exchange  of  TORs/FARs 

If  JCIET  has  recorded  any  TORs  related  to  MIL-STD  conformance  issues,  it  will 
provide  them  to  JITC  for  detailed  analysis.  JITC  will  analyze  these  MIL-STD  TORs  and 
document  the  analysis  in  FARs.  JITC  will  then  provide  their  TORs  with  accompanying 
FARs  back  to  the  JCIET  staff.  If  necessary,  JCIET  will  coordinate  adjudication  of  these 
TORs/FARs  with  system  representatives  to  determine  root  cause  and  problem  isolation. 
The  SAT  will  participate  in  adjudication  of  SIAP-related  TORs.  JCIET,  JITC  and  the 
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system  representatives  will  use  Technical  Interchange  Meetings  and  additional  FAR 
generation  to  vet  any  non-concurrence. 

JITC  will  provide  all  of  their  TORs  related  to  SIAP  issues  and/or  requiring  root 
cause  determination  to  the  JCIET  staff.  JCIET  will  analyze  these  TORs  and  document 
the  analysis  in  FARs.  JCIET  will  then  provide  these  TORs  with  accompanying  FARs 
back  to  JITC  and  system  representatives.  JCIET  will  coordinate  with  system 
representatives  to  determine  root  cause  and  problem  isolation. 

4.3.1 .1 .3  TOR/FAR  Status,  Categorization,  and  Criticality 

JCIET  and  JITC  analysis  will  track  the  status  of  all  TORs/FARS  according  to 
Table  5. 


Table  5.  TOR/FAR  Status 


STATUS 

Further 

Analysis 

Required? 

STATUS  DESCRIPTION 

Open 

Yes 

TOR  is  under  investigation.  Analysis  ongoing. 

No 

A  hardware  or  software  fix  may  have  been  identified  but  the  fix 
has  not  yet  been  verified. 

Closed 

No 

Insufficient  data  and  analysis  of  the  TOR  is  not  feasible. 

No 

After  review  by  cognizant  engineering  activity,  the  TOR  was 
determined  to  be  invalid. 

No 

TOR  has  been  analyzed  by  cognizant  engineering  activity  and 
was  not  reproducible  under  laboratory  conditions  and  analysis 
of  data  tapes  cannot  isolate  the  observed  anomaly. 

No 

Hardware  or  software  fix  has  been  implemented  and  problem 
has  been  verified  as  resolved. 

JCIET,  SAT,  and  JITC  analysts  will  categorize  TORs/FARs  into  categories 
according  to  Table  6.  Given  that  SIAP’s  critical  experiments  are,  in  large  measure,  a 
subset  of  MIL-STD  requirements,  many  TORs/FARs  will  fall  into  both  a  SIAP  and  MIL- 
STD  categories.  These  TORs/FARs  will  be  identified  with  both  categories. 
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Table  6.  TOR/FAR  Categorizations 


FUNCTIONAL  AREA 

CATEGORY  DESCRIPTION 

SIAP  -  Data  Registration 

Uncompensated  data  registration  errors  have  a  significant  adverse 
impact  on  the  SIAP.  Systems  must  implement  standardized  sensor 
registration,  aligning  sensors,  and  inertial  navigation  systems  to  a 
common  geodetic  reference. 

SIAP  -  Automatic  Local-to- 
Remote  Track 
Correlation/Decorrelation 

Not  all  systems  implement  automatic  correlation/decorrelation 
processing,  and  differences  in  the  methods  employed  degrade 
warfighting  capability. 

SIAP  -  Identification  Processing 

Mis-identification  is  frequently  caused  by  erroneous  IFF/SIF-to-track 
association.  Additionally,  differences  in  the  way  automated 
identification  and  data  fusion  systems  behave  adds  to  failure  to  achieve 
commonality. 

SIAP  -  Formation  Tracking  and 
Assessment 

Can  systems  automatically  assign  the  identification  of  a  track  that  has 
been  declared  to  have  a  strength  greater  than  one  to  the  tracks  within 
the  prescribed  formation  “window”,  How  often  will  this  processing 
provide  the  correct  identification?  The  incorrect  capability? 

SIAP  -  Model  and 
Simulation/Stimulation  Fidelity 

Combination  of  modeling/simulation,  HWIL  and  open-air  events  provide 
sufficient  fidelity  to  reflect  operational  system  performance  to  support 
assessment  and  engineering  efforts  to  predict  warfighting 
improvements  of  implementing  ICPs. 

SIAP  -  PPLI  Accuracy 

PPLIs  are  believed  to  be  very  accurate;  however,  due  to  data  link 
latencies  and  the  fact  that  some  navigation  systems  are  not  integrated 
with  the  data  link,  inaccuracies  may  be  present.  These  inaccuracies 
will,  if  present,  degrade  the  network  navigation  solution  and  data 
registration  performance. 

SIAP  -  Multi-Link 

T  ranslation/Forwarding 

There  is  a  need  to  reliably  translate  and  forward  information  from  one 
tactical  data  link  to  another  tactical  data  link  in  ways  that  support  the 
SIAP. 

MIL-STD  -  Violation 

Problem  is  a  violation  of  the  MIL-STD. 

MIL-STD  -  Ambiguity 

Problem  is  the  result  of  poorly  defined  process  in  the  MIL-STD. 

MIL-STD  -  Omission 

Problem  is  not  addressed  by  the  MIL-STD. 

Other 

If  not  in  one  of  the  above  functional  area. 

There  are  a  number  of  different  criticality  categorization  schemes  in  use 
throughout  the  data  link  testing  community.  The  particular  scheme  most  appropriate  for 
use  during  MC  02  and  future  JCIET  events  is  yet  to  be  determined.  The 
JCIET/JITC/SIAP  TF  team  will  collaborate  during  MC  02  to  define  the  most  appropriate 
scheme  useable  by  all  and  apply  that  scheme  to  both  MC  02  and  future  teamed  events. 

Table  7  contains  a  sample  TOR  and  Table  8  contains  a  sample  FAR.  TORs  and 
FARs  will  be  exchanged  between  the  participating  organizations  via  SIPRNET  e-mail 
accounts. 
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Table  7. 


Test  Observation  Report  (TOR) 


Millennium  Challenge  2002  Test  Observation  Report  (TOR) 

Classification: 

(circle  one) 

UNCLAS 

CONF 

SECRET 

System(s)  TOR  is  written 
against: 

TOR  Number: 

Operator  Position: 

Reported  by: 

Phone  #: 

Email: 

Date  of  event: 

Time: 

Zulu  Time: 

Or 

Local  Time: 

Tape  Numbers: 

Description: 

Impact:  (optional) 

TOR  Instructions 

Classification 

Security  classification  of  the  TOR. 

System 

Aircraft,  ship,  or  land  based  site  (TAOC,  CRC,  ICC,  DDG,  etc.)  affected  by 
observed  anomaly. 

TOR  number 

TOR  number  (to  be  assiqned  when  entered  into  tracking  table  or  database) 

Operator  Position 

Watch/test  station  where  the  observation  was  made. 

Reported  by 

Oriqinator  of  the  TOR  and  command. 

Phone  Number 

Phone  number  oriqinator  can  be  reached  at  after  event. 

Tape  Numbers 

Complete  tape  number  for  the  DX  tape  to  use  for  analysis  (include  system,  if 
known).  .  . 

Date  of  event 

Date  of  observation  (MMDD). 

Time 

Time  of  observation.  Desiqnate  either  Zulu  or  Local  Time. 

Description 

A  thorough  description  of  the  observation.  Should  include  system  name  and 
configuration,  scenario  information,  tracks,  identifications,  track  kinematics, 
and  other  information  necessary  to  establish  the  same  environment  as  the 
observation.  Also  include  information  as  to  what  actually  happened  during 
the  observation. 

Impact 

A  brief  description  of  the  operator  impacts  this  deficiency  had  on  the  operator 
or  system  if  not  corrected. 
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Table  8.  Final  Analysis  Report  (FAR) 
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FAR  Instructions 

Classification:  Security  classification  of  the  FAR. 

TOR#:  Represents  the  TOR  number  assigned  by  the  originating  organization 
FAR  Date:  FAR  creation  date. 

FAR  Revision  #:  Represents  the  number  of  analysis  efforts  against  the  TOR. 

Short  Title:  A  very  brief  description  of  the  observation  from  the  referenced  TOR. 

Date  of  Event:  Date  observation  was  made. 

Time  of  Event:  Time  observation  was  made. 

System(s)/IU  #:  Specific  system(s)  affected. 

Program  Version:  Represents  the  combat  system  version  for  the  unit/aircraft  that  originated  the 
deficiency. 

Tapes/Files:  Data  files  used  during  the  analysis  process. 

Analyst  Name:  Person  analyzing  the  deficiency. 

Phone:  Analyst  phone  number. 

E-mail:  Analyst  e-mail  address. 

Analysis  Hours:  Hours  expended  in  the  analysis  of  the  deficiency. 

Status:  Open  or  Closed 

Further  Analysis  Required?:  Yes  or  No 

Test  Objective  Required?  :  Yes  or  No 
Functional  Area:  As  identified  by  Table  6 
TTP  Issue:  Yes  or  No 

Problem  Summary:  A  thorough  description  of  the  observation.  This  information  should  include  what 
actually  happened  during  the  observation. 

Detailed  Sequence  of  Events:  Detailed  description  of  the  events  that  occurred  to  cause  the  deficiency. 
Analysis  Conclusions:  Conclusions  from  the  analysis  effort  to  determine  if  the  deficiency  is  valid  or  if  it 
should  be  closed. 

Proposed  Operation  Workaround:  A  brief  description  of  what  could  be  done  to  work  around  the 
deficiency. 

Operational  Impact:  A  brief  description  of  the  operational  impact  this  deficiency  will  have  on  the 
operator/system  if  not  corrected. 


JITC  will  prepare  TORs  that  are  a  result  of  MIL-STD  violations.  JITC  will  then 
coordinate  further  analysis  by  assigning  the  TOR(s)  to  JCIET  who  will  coordinate  with 
appropriate  system  representatives  for  the  affected  system(s).  JCIET  will  also 
coordinate  with  system  representatives  who  will  use  their  system-specific  analysis  tools 
and  methodologies  to  determine  the  cause  of  the  TOR(s)  and  generate  FAR(s)  that  will 
be  provided  to  JITC. 

JITC,  JCIET,  and  system  representatives  will  discuss  and  adjudicate  all 
TORs/FARs  analysis  results.  Once  the  TORs  are  fully  adjudicated,  JITC  will  create  a 
synopsis  of  the  anomaly  and  analysis  based  on  the  information  contained  in  the  TORs 
and  FARs  and  subsequent  discussions.  JITC,  in  close  coordination  with  JCIET,  will 
include  a  MIL-STD  conformance  section  within  its  report  for  MC  02. 
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4.4  Data  Reduction 


Each  organization  (JCIET  and  JITC)  will  perform  data  reduction  and  analysis 
using  current  organizational  methodologies.  Each  organization  is  responsible  for 
ensuring  the  integrity  of  data  reduction  results  that  it  publishes  within  its  own  report  to 
the  USJFCOM. 

4.5  Data  Management  and  Storage 

Each  organization  will  perform  its  own  data  archival.  Classified  data  transfer 
between  the  two  organizations  will  be  performed  via  mail  (including  FedEx)  or  SIPRNET 
for  softcopies,  depending  on  the  size  of  the  data  transfer  to  be  accomplished.  JCIET 
collected  data  will  not  be  transferred  to  JITC  without  the  consent  of  the  system  program 
office.  Table  9  provides  the  contact  information  for  data  storage. 


Table  9.  Data  Management  and  Storage  Worksheet 


Name  of  Person/Organization  providing  information 

System  Identification  (incl.  computer  program  version) 

Description  of  data  collected  and  available,  including  POC  and  accessibility  information 

Name  of  Person/Organization  providing  information 

System  Identification  (incl.  computer  program  version) 

Description  of  data  collected  and  available,  including  POC  and  accessibility  information 


4.6  Limitations 

MC  02  is  limited  in  its  SAT  support,  both  in  manning  and  in  funding.  The  SAT  will 
not  be  providing  on-site  analysis  support  at  MC  02,  and  will  provide  minimal  support  to 
the  post-event  root-cause  analysis  effort  (time  and  funding  permitting).  The  emphasis 
of  this  event  is  on  developing  the  process  (relationships,  roles)  between  JITC  and 
JCIET  to  support  JDN  assessments. 


5.  DATA  ANALYSIS  PROCESS 

Both  JCIET  and  JITC  have  existing  data  processing  and  analysis  methodologies. 
This  document  does  not  address  each  organization’s  processing/analysis  methodology, 
but  is  intended  to  describe  the  management  of  data  that  are  shared  between  the  two 
organizations  to  support  joint  analysis. 
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Both  JCIET  and  JITC  will  conduct  post-event  analysis.  JITC  will  analyze  data  to 
support  analysis  of  overall  JDN  performance  and  CJCSM  61 20.01  B  conformance  for 
both  live  and  simulated  systems  that  produce  data  onto  the  JDN  as  detailed  in  JITC’s 
MC  02  Interoperability  Support  Plan.  JCIET  will  provide  SIAP  attributes  using  criteria 
and  methodology  defined  in  SIAP  SE  Technical  Report  2001-001  and  SIAP  SE 
Technical  Report  2001-003. 

During  the  week  of  9  through  13  September  2002  (this  is  a  target  date), 
representatives  of  USJFCOM,  JCIET,  JITC,  and  members  of  the  SAT  will  meet  and 
prioritize  areas  of  focus  for  post-event  analysis. 

Section  5.1  discusses  the  post-event  MIL-STD  601 6A  and  CJCSM  61 20.01  B 
compliance  post-event  efforts.  Section  5.2  discusses  the  SIAP-related  analysis.  Section 
5.3  discusses  the  Joint  analysis  improvement  aspect  of  the  post-event  analysis. 

5.1  MIL-STD  601 6A  and  CJCSM  6120.01  B  Compliance 

JITC  will  follow  the  process  described  in  Section  3.1  to  address  the  MIL-STD 
601 6A  and  CJCSM  61 20.01  B  compliance. 

JITC  shall  perform  analysis  of  system  performance  related  to  MIL-STD  issues. 
JITC  will  review  TORs  generated  during  on-site  analysis  (during  event  execution)  and 
as  a  result  of  post-event  analysis  by  both  JCIET  and  JITC  staffs. 

JITC  shall  perform  post-event  analysis  of  critical  simulation  systems  identified  by 
USJFCOM  to  support  seamless  operations  in  the  JDN  (i.e.,  the  extent  to  which  the 
simulation  system  behaves  like  its  real-world  counterpart  by  examining  the  data 
contributed  to  the  JDN  by  the  simulation  system).  A  primary  goal  of  this  analysis  effort 
is  to  assist  USJFCOM  in  determining  the  extent  to  which  simulation  systems  must: 

a.  Comply  with  MIL-STD  601 6A. 

b.  Replicate  the  behavior  of  their  real-world  counterparts. 


5.2  IADS  Performance  Assessment 
5.2.1  Data  Availability  Matrix 

In  conjunction  with  JITC  and  JCIET,  SAT  members  (time  and  funding  permitting) 
will  be  responsible  for  reviewing  the  data  recorded  and  identify  the  time  sets  to  be 
analyzed. 
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5.2.2  Critical  Experiments 

In  conjunction  with  JITC,  SAT  members  (time  and  funding  permitting)  will  review 
the  data  availability  matrix  that  is  developed  and  identify  events  where  sufficient  data  to 
support  the  critical  experiments  exists.  The  tables  the  SIAP  Standard  DMAP  (Section 
5.2.1)  provide  a  cross-reference  of  the  critical  experiments  to  the  system  level  data 
collection  capability/points. 

5.2.3  SIAP  Metrics 

Operational  shortfalls  (leakers,  fratricides,  reduced  mission  effectiveness)  are 
often  caused,  at  least  in  part,  by  deficiencies  in  the  SIAP  presented  to  the  operator.  For 
MC-02,  JCIET  will  lead  an  assessment  of  the  quality  of  the  SIAP  in  terms  of  SIAP 
attributes,  defined  in  the  SIAP  SE  Technical  Reports  2001-001  and  2001-003.  These 
mathematically  defined  SIAP  attributes  were  developed  to  support  calculation  of  the  key 
performance  parameters  in  the  TAMD  and  Combat  Identification  (CID)  CRDs.  A  JCIET- 
led  team  of  individuals  from  JCIET,  CNA,  NWAS,  and  PEO  AMD  will  perform  this 
analysis. 

5.2.4  Root-Cause  Analysis 

After  calculation  of  the  attributes,  JCIET  will  lead  an  effort  to  determine  the  root 
causes  of  SIAP  deficiencies.  For  example,  for  instances  when  the  picture  is  not 
common  at  all  systems;  the  underlying  reasons  for  these  commonality  deficiencies  will 
be  determined.  To  do  this,  tracks  available  to  the  operator  for  surveillance  systems  in 
the  JDN  will  be  matched  to  Time  space  Position  Indicator  (TSPI).  The  results  of  this 
match  will  assign  tracks  held  by  systems  to  one  or  more  TSPI  aircraft.  Comparisons 
between  systems  and  between  a  system  and  TSPI  can  then  be  made  in  accordance 
with  SIAP  SE  defined  attributes. 

JITC  and  the  SAT  will  provide  contact  information  for  individuals  who  can 
contribute  to  the  root-cause  analysis  effort  as  shown  in  Table  10. 


Table  10.  Root-Cause  Analysis  Participant  Points-of-Contact  Worksheet 


Name  of  Person/Organization  providing  analysis 

System  Identification  (incl.  computer  program  version) 

System  issues  analysis  (including  TTPs) 

Name  of  Person/Organization  providing  analysis 

System  Identification  (incl.  computer  program  version) 

System  issues  analysis  (including  TTPs) 
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5.2.4.1  Events  of  Interest 

Events  of  Interest  (EOls)  will  be  identified  on-site  at  the  event  as  well  as  during 
the  root-cause  phase  of  the  analysis  after  the  event. 

5.2.4.2  Test  Observation  Reports 

TORs  related  to  IADS  performance  assessment  will  be  reviewed  and  adjudicated 
during  and  after  the  event  as  discussed  in  Section  4.3.1. 1.  JITC  and  SAT 
representatives  will  coordinate  to  develop  a  schedule  for  adjudicating  TORs  and 
identifying  SIAP-related  lessons  learned  to  the  SIAP  SE  TF  for  inclusion  in  the  SIAP 
lessons  learned  database. 

5.3  Joint  Analysis  Improvement 

The  joint  analysis  effort  to  be  conducted  by  JCIET  and  JITC  is  hampered  by  the 
short  planning  cycle  prior  to  the  conduct  of  MC  02.  Throughout  the  data  collection  and 
analysis  effort,  both  organizations  will  document  any  factors  that  limited  joint  analysis  by 
both  participating  organizations  as  well  as  those  factors  that  impeded  analysis  efforts  by 
the  individual  organization.  JITC  will  coordinate  their  findings  with  applicable  members 
of  the  SAT  prior  to  publishing  their  report.  The  goal  of  this  effort  is  to  document 
improvements  in  the  process  that  can  be  implemented  prior  to  commencing  activities  for 
the  next  effort  after  MC  02  (ROVING  SANDS  2003  is  a  candidate). 

5.3.1  Development/Analysis  Phases 

Throughout  the  data  collection  and  analysis  effort,  both  organizations  will 
document  any  factors  that  limited  joint  analysis  by  both  participating  organizations  as 
well  as  those  factors  that  impeded  analysis  efforts  by  the  individual  organization.  These 
factors  include  any  organizational,  technical,  or  procedural  inhibitors  to  the  joint  analysis 
process,  and  any  workarounds  developed. 

5.3.2  Final  Reporting  Phase 

The  purpose  of  this  phase  is  to  report  any  organizational,  technical,  or  procedural 
inhibitors  to  the  reporting  process  and  to  identify  any  workarounds  developed. 


6.  REPORTING 

6.1  Quick-Look  Report 

JITC  will  provide  a  quick-look  report  on  the  execution  analysis  by  27  September 
02  (target  date).  It  will  include  a  characterization  of  the  Joint  Data  Network  (JDN), 
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6.2  Technical  Reports 


JITC  will  generate  and  provide  Technical  Reports  (TRs)  for  the  MC  02  event. 
Throughout  the  data  collection  and  analysis  effort,  both  organizations  will  document  any 
factors  that  limited  joint  analysis  by  both  participating  organizations  as  well  as  those 
factors  that  impeded  analysis  efforts  by  the  individual  organization.  Both  JCIET  and 
JITC  will  coordinate  their  findings  prior  to  publishing  any  individual  reports.  The  goal  of 
this  effort  is  to  document  improvements  in  the  process  that  can  be  implemented  prior  to 
commencing  activities  for  the  next  effort  after  MC  02  (ROVING  SANDS  2003  is  a 
candidate).  The  final  report  will  document  an  assessment  of  the  limited  TOR  process 
conducted  during  MC  02,  including  any  limitations  or  changes  to  the  TOR  process 
proposed  in  Section  4.3. 1.1. 

6.3  Lessons  Learned 

JCIET  and  JITC  will  work  with  the  SAT  to  generate  lessons  learned  from  the  MC 
02  event.  These  lessons  will  include  issues  with  logistics,  planning,  execution,  and 
analysis,  with  a  focus  on  the  teaming  process  developed  between  JCIET  and  JITC. 
Lessons  learned  will  also  capture  limitations  of  MC  02  as  an  appropriate  SIAP  venue. 
JCIET  and  JITC  will  work  with  the  SAT  to  identify  and  document  what  critical 
experiments  and  post-event  analysis  activities  can  and  cannot  be  conducted  at  MC  02. 

6.4  Unresolved  Issues 

Any  unresolved  issues  will  be  documented  and  included  in  the  appropriate 
reports. 
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APPENDIX  A 


ABT 

ADSI 

af 

AFB 

AWACS 

BLUFOR 

CAOC 

CID 

CJCSM 

CM 

CTS 

disa 

DMAP 

DOT&E 

DSN 

ECM 

EOI 

EW 

FAR 

GMT 

HLA 

IADS 

ID 

IFF/SI 

JCIET 

JDN 

JICO 

JITC 

JMTOP 


ACRONYMS 


Air  Breathing  Threat 

Air  Defense  System  Integrator 

Air  Force 

Air  Force  Base 

Airborne  Warning  and  Control  System 
Blue  Forces 

Combined  Air  Operations  Center 
Combat  identification 

Chairman  of  the  Joint  Chiefs  of  Staff  Manual 
Cruise  Missile 
Central  T rack  Stores 

Defense  Information  Systems  Agency 
Data  Management  and  Analysis  Plan 
Director,  Operational  Test  and  Evaluation 
Defense  Switched  Network 

Electronic  Countermeasures 
Event  of  Interest 

Electronic  Warfare 
Final  Analysis  Report 

Greenwich  Mean  Time 

High  Level  Architecture 

Integrated  Air  Defense  System 
Identification 

Identification  Friend  or  Foe 

Joint  Combat  Identification  Evaluation  Team 
Joint  Data  Network 
Joint  Interface  Control  Officer 
Joint  Interoperability  Test  Command 

Joint  Multi-Tactical  Digital  Information  Link  Operation  Procedures 
(CJCSM  61 20.01  B) 
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JOCAT 

Joint  Operational  C4I  Assessment  Team 

LMS-16 

LOB 

Link  Monitoring  System-16 

Line  of  Bearing 

MC  02 

MIL-STD 

mil 

MILLENNIUM  CHALLENGE  2002 

Military  Standard 

Military 

NACTS 

Nellis  Air  Combat  Training  System 

OPFOR 

Opposing  Forces 

POC 

Point  of  Contact 

PPLI 

RF 

Precise  Participant  Location  and  Identification 

Radio  Frequency 

SE 

SIAP 

SIPRNET 

System  Engineer 

Single  Integrated  Air  Picture 

SECRET  Internet  Protocol  Router  Network 

TBM 

TDDS 

TDIMF 

TIAC 

TIBS 

TMD 

TOR 

TRAP 

Theater  Ballistic  Missile 

TRAP  Data  Dissemination  System 

Tactical  Data  Intercomputer  Message  Format 

TMD  Interoperability  Assessment  Capability 

Tactical  information  Broadcast  Service 

Theater  Missile  Defense 

Test  Observation  Report 

Tactical  Receive  Equipment  and  Related  Applications 

USJFCOM 

United  States  Joint  Forces  Command 
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Czifra  Cynthia  B  CONT  NSSC 


From: 

Sent: 

To: 

Cc: 

Subject: 


127-B-30 . 1 
Way  ahead 

*  Use  Integrated  Architecture  to  build  reference  implementation  and 
verification/validation  capability 

-  FFRDC  (a  la  HLA  RTI  model) 

-  Industry  (may  cause  future  competition  problems) 

*  Reference  implementation 

-  Leverages  significant  work  already  underway  in  existing  disconnected  and  asynchronous 
acquisition  programs 

-  Describes  functionality  that  must  be  provided  by  industry  in  final  system  configurations 

-  Used  by  industry  as  baseline  for  capability  and  performance  (industry  has  option  of 
using  reference  implementations  as- is,  if  performance  is  adequate  to  meet  operational 
requirement) 

-  Modular,  component -based  approach  allows  for  competition  at  component  level 

-  Built  on  industry- standard,  open  application  programming  interfaces  (APIs) 

-  Built  using  JDEP  (scaleable  distributed  development  environment,  with  access  to  existing 
systems) 

-  Maintained  as  part  of  Integrated  Architecture 

-  Uses  approach  employed  by  NIST  for  crytographic  algorithms  and  Internet  community  for 
internetworking  protocols 

*  Verification/validation  capability 

-  Built  in  partnership  with  DOT&E  to  help  meet  OT&E  needs 

-  Built  in  Partnership  with  DISA  to  help  meet  interoperability  certification  needs 

-  Maintained  in  lock-step  with  reference  implementation  as  part  of  Integrated  Architecture 


Wilson  Jeff  W  CAPT  NSSC 
Tuesday,  August  27,  2002  3:37  PM 
Czifra  Cynthia  B  CONT  NSSC 
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